Towards student/teacher learning in sequential decision tasks

Significant advances have been made in allowing agents to learn, both autonomously and with human guidance. However, less attention has been paid to the question of how agents could best teach each other. For instance, an existing robot in a factory should be able to instruct a newly arriving robot, even if it is from a different manufacturer, has a different knowledge representation, or is not optimal itself. This work investigates teaching methods in sequential decision tasks. In particular, we consider a reinforcement learning studentagent that must learn from 1) autonomous exploration of the environment and 2) the guidance of another teacher-agent. In order to minimize inter-operability requirements, the teacher and student are presumed not to know each others’ internal workings; teachers can only help students by suggesting actions. Furthermore, the teacher may have limited expertise in the student’s task and should be careful not to over-advise the student. Our primary question: how should the teacher decide when to give advice? This teaching context is related to the more well-studied problem of transfer learning [5], in which an agent uses knowledge from a source task to aid its learning in a target task, but differs in that we do not assume agents can directly access each others’ internal knowledge. Another related area is learning from experts [1, 3], where agents may imitate experts or ask for their advice. Our approach differs because control is given to the teacher, rather than the student, and we focus on non-expert teachers. Our hope is that this paper enables and inspires the agents community to develop further methods by which agents can teach other agents.